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(54) Region-of-interest setting apparatus for respiration monitoring and a respiration 
monotoring system 



(57) A region-of-interest (ROI) setting apparatus for 
respiration monitoring and a respiration monitoring sys- 
tem are capable of setting an ROI in an automatic man- 
ner. Absolute values of subtractions between every 
successive pairs of frame images produced by a CCD 
camera 2 for a period of time equal to half a respiration 
period are calculated, then summed and stored. Based 
on the change information thus calculated and stored, 
the position and size of each changed region is calcu- 
lated. Provisional regions are set from the changed 
regions successively from the greatest to the smallest 



one thereof. In each of these provisional regions, 
processing is carried out to determine whether a grey 
level histogram indicative of distribution of the number of 
pixels for respective grey level values includes a twin 
peak characteristic having twin peaks each equal to or 
higher than a predetermined value, and whether the 
area of the corresponding changed region is equal to or 
greater than a predetermined value. One of the provi- 
sional regions that satisfies these conditions is set as an 
ROI. 
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Description 



[0001] The present invention relates to a respiration 
monitoring system which is capable of continuously 
monitoring the respiratory state of a person, involving, 
for instance, visual sensing of a patient in a non-restric- 
tive manner. The invention also relates to apparatus for 
setting a region of interest for respiration monitoring. 
[0002] In recent years, the number of persons of 
advanced age has increased rapidly and it is desired to 
increase the number of staff who take care of bedridden 
elderly persons with attendant increasing personnel 
costs and medical care expenditure. This is an object of 
great public concern. 

[0003] Respiration monitoring systems have been 
developed which are capable of continuously monitor- 
ing the respiratory state of a bedridden person through 
visual sensing in a non-restrictive manner. 
[0004] Fig. 1 3 shows a typical conventional respiration 
monitoring system as disclosed in "Automatic Measure- 
ments of the Number of Respirations under Complete 
Non-restrictions by a Visual Sensing System" reported 
in the Proceedings of the 16th Bio-Mechanism Society 
Conference held on November 25, 1995 by the Bio- 
Mechanism Society (pages 279 to 282). This respiration 
monitoring system includes a CCD camera 2 for imag- 
ing or photographing the chest of a patient 1 as an 
object to be imaged who is lying on a bed, and an image 
processor 3 connected to an output of the CCD camera 
2 for successively extracting an inter-image difference 
or subtraction (i.e., subtraction between every two suc- 
cessive images) of continuous motion pictures at a con- 
stant rate. Predetermined image processing steps are 
performed so as to obtain or calculate a difference com- 
ponent thereby to measure the number of respirations 
per unit time. The system also includes a monitor 4 con- 
nected to an output of the image processor 3 for display- 
ing a visual indication of the detected data including the 
measured number of respirations per unit time, the state 
of respiration and/or for displaying an abnormal state in 
the case of an abnormality taking place. A setting input 
stage 5 allows an operator to input various settings such 
as a processing region in the form of a region of interest 
(hereinafter simply referred to as "ROI") of the image 
processor 3, a subtraction interval (a) in the image 



[0005] Fig. 1 4 is a flow chart showing operation (i.e., 
respiration counting operation) of the above-mentioned 
conventional respiration monitoring system. Firstly in 
step S1 , the image processor 3 successively receives 
image data from the CCD camera 2, and in step S2, it 
performs signed (+, -) subtraction of pixels between 
every adjacent two images that are spaced from each 
other by a predetermined frame interval (a). Subse- 
quently, in step S3, the image processor 3 performs sur- 
face integration within a ROI as follows. The extracted 
image data are subjected to signed subtraction (i.e., 
subtraction with a plus or minus sign) to provide bina- 



rized images, from which an area of variations for each 
image is calculated. Each area thus obtained is then 
surface integrated in terms of brightness to provide a 
signed rate of change for each changed portion. In step 
5 S4, the signed rate of change is processed in a serial 
manner each time the CCD camera 2 takes one frame 
to produce a result on which respiration counting is car- 
ried out in step S5. 

[0006] In this manner, the conventional respiration 
10 monitoring system can monitor the state of respiration 
through visual sensing in a non-restrictive and continu- 
ous manner. 

[0007] With this prior system, however, a supervisor 
who also acts as an operator has to input manually a 

15 specific region of interest (ROI) setting among the pic- 
tures or images photographed by the CCD camera 2, for 
which region image processing is to be performed for 
measuring or counting the number of respirations per 
unit time, while watching the monitor 4, referred to 

20 above. 

[0008] Thus, with the conventional system in which 
the ROI setting is input manually, if, for instance, a 
patient to be monitored or photographed has moved to 
displace the ROI, it is difficult to measure the number of 

25 respirations per unit time in an accurate manner. In 
practice respiration measurement becomes impossible 
when such a situation is left unattended. As a result, it is 
impossible to measure automatically the respiration rate 
for a long period of time. On the other hand, if the devi- 

30 ation or displacement of the ROI (i.e., movement of the 
person) is monitored by a supervisor to allow long-term 
accurate respiration counting, personnel expense and 
the amount of labour cannot be reduced. 
[0009] Moreover, in this case, a change in the value of 

35 surface integration becomes small depending upon the 
ROI setting, so the accuracy of respiration counting can 
be reduced due to noise or other interference, or 
becomes difficult to recognise inhalation and exhalation 
of the person on a monitor. 

40 [0010] Furthermore, with the conventional respiration 
monitoring system, the frame interval a for subtraction 
is also set and input by the operator, so similarly to the 
setting of the ROI, this gives rise to difficulty in long-term 
respiration counting. The frame interval a is input by the 

45 operator based on his or her experience, so the accu- 
racy of respiration counting depends on the setting of a 
and sometimes decreases due to an inappropriate set- 
ting, with the result that recognition of inhalation and 
exhalation on the monitor becomes difficult. 

so [0011] Further, the conventional respiration monitor- 
ing system calculates the rate of change in the darkness 
or grey level within the ROI using an inter-image sub- 
traction with a plus or minus sign, and hence, if an 
increase and a decrease of brightness within the ROI 

55 are equal to each other in their areas, it is determined 
that there is no change in the rate. 
[001 2] According to a first aspect of the present inven- 
tion, there is provided region-of-interest setting appara- 
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tus for respiration monitoring comprising: 

imaging means (such as a camera) for imaging an 
object to be monitored; 

5 

provisional-region setting means for extracting 
changed portions between successive images 
taken by the imaging means and setting an optimal 
one of the thus extracted changed portions as a 
provisional region; 10 

determining means (e.g. an optimal-ROI determin- 
ing stage) for determining whether the provisional 
region set by the provisional-region setting means 
can be adopted as a changed region for respiration is 
counting; and 

region-of-interest setting means (e.g. an ROI set- 
ting and storing stage) for setting the provisional 
region as a region of interest for respiration count- 20 
ing when the determining means determines that 
the provisional region can be adopted as a changed 
region for respiration counting. 

With the above arrangement, it is possible to set the ss 
region of interest in an automatic manner. 
[0013] In a preferred embodiment, the provisional- 
region setting means comprises: 

subtracted-image storing means (e.g., an original 30 
image memory and a subtraction and summation 
circuit) for calculating absolute values of subtrac- 
tions between every successive two of a plurality of 
frame images taken during at least half a respira- 
tion period, and for summing and storing the abso- 35 
lute values of subtractions of the images thus 
obtained for predetermined frames; and 

region-of-interest candidate setting means for cal- 
culating the position and size of each of the 40 
changed regions based on brightness information 
of each subtracted image stored in the subtracted- 
image storing means and setting at least the great- 
est among the changed regions as a provisional 
region. 45 

[001 4] With the above arrangement, for a candidate of 
the region of interest, a single region can be selected in 
which there can be seen changes over the largest area, 
and hence it is possible to select as an ROI candidate a so 
region for which a sufficient respiration measuring or 
counting accuracy can be ensured in the presence of 
noise. 

[001 5] In another preferred embodiment, the region- 
of-interest candidate setting means (e.g., a candidate- 55 
region extracting stage) is arranged to calculate the 
position and size of each of the changed regions based 
on brightness (change) information of each subtracted 



image stored in the subtracted-image storing means 
and to select an optimal one of the changed regions 
successively from the greatest to the smallest thereof 
as a provisional region. 

[001 6] With the above arrangement, in cases where a 
large region is not used for setting a region of interest, 
the next candidate can be retrieved, thus obviating the 
necessity of again performing a provisional region set- 
ting operation. This leads to an enhancement in 
processing. 

[0017] In a further preferred embodiment, the deter- 
mining means determines whether in the provisional 
region set by the provisional-region setting means, a 
grey level histogram indicative of distribution of the 
number of pixels for respective grey level values 
includes a twin peak characteristic having twin peaks 
each equal to or higher than a predetermined value. In 
one embodiment, the determining means comprises an 
optimal-ROI determining stage, and the predetermined 
value is set in such a manner that peaks of the twin- 
peak characteristic can be detected in the presence of 
noise. 

[001 8] In this way, on the basis of the grey level histo- 
gram having a vertical difference (i.e., difference 
between the highest and lowest points) not less than a 
predetermined value, it is possible to select, among the 
ROI candidates, an optimal region of interest in which a 
sufficient difference in contrast can be detected to pro- 
vide a satisfactory accuracy in optical measurements. In 
this case, since the twin peak characteristic is a 
required condition, reliability in maintaining measuring 
accuracy can be further enhanced. 
[001 9] In a yet further preferred form the determining 
means (an optimal-ROI determining stage) determines 
whether the area of the changed region is equal to or 
greater than a predetermined value. This allows not 
only selection of an optimal region of interest for which 
sufficient measuring accuracy can be obtained in the 
presence of noise, but also easily to recognise varia- 
tions in exhalation and inhalation of the object such as a 
patient on a monitor for example. 
[0020] In a still further preferred form of, when the 
determining means (an optimal-ROI determining stage) 
determines that the provisional region set by the provi- 
sional region setting means cannot be adopted as a 
changed region for respiration counting, the provisional 
region setting means again performs the provisional 
region setting. Accordingly, the process of setting a 
region of interest is carried out repeatedly until an opti- 
mal region of interest is set, thereby increasing reliability 
in ROI setting. 

[0021] In a further preferred form, the apparatus fur- 
ther comprises region-of-interest setting starting-condi- 
tion determining means (e.g., an object detecting stage) 
for detecting, as a condition for starting the setting of a 
region of interest, whether the object to be monitored 
has come into a field of view of the imaging means. 
[0022] With this arrangement, even if the patient has 
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moved away from the bed, when the patient returns, a 
region of interest is automatically set once again, so 
respiration monitoring can be resumed in an automate 
manner. 

[0023] According to a second aspect of the invention, 5 
there is provided a respiration monitoring system com- 
prising: 

imaging means for imaging an object to be moni- 
tored; 10 
setting means for setting a region of interest for res- 
piration counting in images taken by a imaging 
means; 

change detecting means (e.g., an ROI image 
extracting stage, a previous-frame memory, a is 
frame-memory selecting stage, an inter-image sub- 
traction stage and a surface integration stage 25) 
for calculating absolute values of subtractions 
between successive images produced by the imag- 
ing means in the region of interest set by the setting 20 
means and for summing up the absolute values of 
subtractions for every predetermined number of 
frames; 

counting means (e.g., a serial data storing stage 
and a respiration calculating stage) for counting the 25 
number of respirations per unit time based on a 
trend of time-series changes obtained by the 
change detecting means. 

[0024] With the above arrangement, the absolute val- 30 
ues of subtractions between successive images are 
summed for every predetermined number of frames, so 
that even if the brightness in the region of interest 
changes with the area of increasing brightness equal to 
the area of decreasing brightness, such changes can be 35 
detected correctly, thus preventing errors in measure- 
ments which would otherwise result from failure to 
detect the changes as in the prior art. 
[0025] In a preferred form, the system further com- 
prises condition determining means for determining, 40 
based on a frequency analysis of a trend of time-series 
data, whether the state of respiration is normal. 
[0026] With this arrangement, discrimination can be 
clearly made as to whether the trend of time-series data 
is generated based on noise and the like, or on normal 45 
respiration. 

[0027] In another preferred form, the system further 
comprises existence determining means for determin- 
ing whether the object to be monitored is outside a pre- 
determined imaging range. In this way it can be so 
determined whether, for example, a patient to be moni- 
tored has moved out of a range to be monitored. This 
may happen when, for example, the patient gets out of 
bed for lingering or some other reason, or falls down 
from the bed. This serves to improve reliability in respi- 55 
ration monitoring. 

[0028] In a further preferred form, the respiration 
counting means counts the number of respirations per 
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unit time by measuring the number of peaks in a change 
of a trend in the time series data, and wherein a thresh- 
old is used which is variably set for each peak for count- 
ing thereof. Accordingly, the number of respirations per 
unit time can be reliably counted. That is, the magnitude 
of each peak occurring in the trend of the time-series 
data generally changes, and in such a case, if the 
number of peaks is counted with a constant threshold, 
there will be a problem that peaks smaller than the 
threshold cannot be counted. To counter this, the 
threshold is set according to the magnitude of each 
peak, so that the number of peaks can be counted cor- 
rectly irrespective of variations in the peak magnitude. 
[0029] In a still further preferred form, the system fur- 
ther comprises frame-interval setting means (typically a 
frame-interval setting stage) for setting the predeter- 
mined number based on the summed value calculated 
by the change detecting means or the number of respi- 
rations per unit time counted by the counting means in 
such a manner as to emphasize changes in grey levels. 
With this arrangement, measuring accuracy can be 
improved, and observation of changes on a monitor for 
example can be effected with ease. 
[0030] In a yet further preferred form, the means for 
setting a region of interest for respiration counting in 
images taken by the imaging means comprises the 
region-of-interest setting means (an ROI setting stage) 
of the region-of-interest setting apparatus for respiration 
monitoring as set out above. With the above arrange- 
ment, the entire process of respiration monitoring 
including the setting of a region of interest can be auto- 
mated so that respiration of the patient can be moni- 
tored for an extended period of time without requiring 
any human monitoring.. 

[0031] In a further preferred form, the provisional 
region setting means and/or the determining means 
operate during respiration counting. Therefore, in cases 
where the ROI needs to be set again during respiration 
counting, such new setting can be readily made imme- 
diately, thus avoiding interruption of respiration monitor- 
ing for a long period of time. 

[0032] The invention will now be described by way of 
example with reference to the drawings in which:- 

Fig. 1 is a block diagram showing the overall 
arrangement of a first respiration monitoring system 
in accordance with the invention; 

Fig. 2 is a flow chart showing the overall operation 
of the respiration monitoring system; 

Fig. 3 is a block diagram showing details of an ROI 
setting stage and a respiration measuring stage. 

Rg. 4 is a flow chart showing the operation of the 
ROI setting stage; 

Fig. 5 is a representation of image change informa- 
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tion obtained by a subtracting and summing circuit 
of the system; 

Fig. 6 is a grey level histogram; 

Fig. 7 is a flow chart showing respiration counting; 

Fig. 8 is a time chart showing the respiration speed 
or rate in terms of the absolute value of subtraction; 

Fig. 9 is a block diagram showing an optimal-ROI 
determining stage of a second respiration monitor- 
ing system in accordance with the present inven- 
tion; 

Fig. 10 is a block diagram showing a respiration 
rate calculating stage of a third respiration monitor- 
ing system in accordance with of the invention: 

Fig. 1 1 is a block diagram showing an exemplary 
method for frequency analysing time-series data in 
a fourth respiration monitoring system in accord- 
ance with the invention; 

Fig. 12 is a block diagram showing the overall 
arrangement of a fifth respiration monitoring system 
in accordance with the invention: 

Fig. 13 is a block diagram showing a conventional 
respiration monitoring system; and 

Fig. 14 is a flow chart showing the operation of the 
conventional respiration monitoring system. 

EMBODIMENT 1 

[0033] Referring to Fig. 1 , a respiration monitoring 
system comprises a CCD camera 2 for sensing or imag- 
ing the chest or thereabouts of a target object such as 
patient, who is being cared for, a region-of-interest 
(ROI) setting stage 10 connected to an output of the 
CCD camera 2 for setting a region of interest, a respira- 
tion counting stage 20 connected to an output of the 
ROI setting stage 10 and the output of the CCD camera 
2 for counting the number of respirations per unit time in 
the region of interest (ROI) which is set by the ROI set- 
ting stage 10, an abnormality determining stage 30 con- 
nected to an output of the respiration counting section 
20 for determining whether the number of respirations 
per unit time measured is normal or abnormal, and an 
object detecting stage 40 connected to an output of the 
abnormality determining stage 30 and the output of the 
CCD camera 2 for detecting the target object to be mon- 
itored. 

[0034] The object detecting stage 40 detects the tar- 
get object coming into the view in the CCD camera 2 
upon setting of the ROI by continuously detecting a 
large displacement of the target object on a picture 
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image for a predetermined period of time, and gener- 
ates a detection or output signal which acts as a trigger 
to actuate the ROI setting stage 10. Also, when the tar- 
get object temporarily moves away from a bed or the 

5 like, the abnormality determining stage 30 determines 
an abnormality in the number of respirations per unit 
time measured, and generates a corresponding output 
signal, based on which the object detecting stage 40 is 
started to prepare for watching the bed or thereabouts 

w and again detects the target object when it has returned 
to the bed, thus reactuating the ROI setting stage 10. 
[0035] Referring to Fig. 2, in step S1 1 , the system is 
actuated, and in step S12, the object detecting stage 40 
starts to detect the presence and absence of a target 

15 object, i.e., whether a target object is on the bed. Upon 
detection of the target object, the ROI setting stage 10 
sets a region of interest (ROI) in step S13. Upon setting 
of the ROI the ROI setting stage 10 sets a candidate for 
the ROI and determines the validity or feasibility thereof. 

20 If it is determined that the setting of the ROI candidate is 
invalid, the control process or flow returns to the exist- 
ence detecting step S12, and the above operations in 
steps S12 and S13 are repeatedly carried out until a 
valid ROI candidate is obtained becomes valid. 

25 [0036] If the ROI candidate setting is determined to be 
valid in step S13, the flow then goes to step S14 where 
respiration counting is carried out. Subsequently, in step 
S15, if it is determined that the number of respirations 
per unit time or the movement of the target object (e.g., 

30 body movement of a person to be monitored) is abnor- 
mal, there may be the possibility that the target object 
be away from the bed, and thus, the control process 
returns to the existence detecting step S12, whereas if 
the determination in step S15 is otherwise (i.e., abnor- 

35 mal), an alarm is generated by the abnormality deter- 
mining stage 30. 

[0037] Fig. 3 shows in block form the details of the ROI 
setting stage 10 and the respiration counting stage 20 of 
the respiration monitoring system. 

40 [0038] The ROI setting stage 1 0 comprises an original 
image memory 1 1 connected to an output of a preproc- 
essing circuit 6 that performs waveform shaping and the 
like, for continuously taking in the image obtained by the 
CCD camera 2 for a period of time just half the respira- 

45 tion period. A subtracting and summing circuit 12 is con- 
nected to an output of the original image memory 1 1 for 
extracting a changed portion of the image stored 
therein, and a candidate-region extracting stage 13 is 
connected to an output of the subtracting and summing 

so circuit 12 for labelling the summed image obtained by 
the subtracting and summing circuit 12 to provide a can- 
didate region for the ROI. An optimal-ROI determining 
stage 14 is connected to an output of the candidate- 
region extracting stage 13 and the output of the original 

55 image memory 1 1 for determining whether the region 
(image region) extracted by the candidate-region 
extracting stage 13 can be used as the ROI and an ROI 
setting and storing stage 1 5 is connected to an output of 
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the optimal-ROI determining stage 14 for setting as the 
ROI the image region determined to be appropriate or 
optimal for the ROI by means of the optimal-ROI deter- 
mining stage 14. 

[0039] Here, it is to be noted that an operation control- 
ler (not shown) is connected to an output of the object 
detecting stage 40 shown in Fig. 1 . 
[0040] The respiration counting stage 20 comprises 
an ROI image extracting stage 21 connected to the out- 
put of the preprocessing circuit 6 and the output of the 
ROI setting and storing stage 15 for extracting from the 
image data obtained by the CCD camera 2 only those 
image data which lie within the ROI and a previous- 
frame memory 22 connected to an output of the ROI 
image extracting stage 21 for storing ROI image data for 
n frame images. An inter-image subtraction stage 24 is 
connected to an output of the ROI image extracting 
stage 21 and an output of a frame memory selecting 
stage 23 (described later) for performing subtraction 
between images spaced from each other a distance of 
a frames. A surface integrates stage 25 connected to an 
output of the inter-image subtraction stage 24 for sur- 
face integrating the values of the inter-image subtrac- 
tions thereby obtained. 

[0041] The respiration counting stage 20 comprises a 
time-series-data storing stage 26 connected to an out- 
put of the surface integration stage 25 for storing, as 
serial data, the values of surface integration obtained by 
surface integrating the inter-image subtraction in a time 
serial manner for a predetermined period of time, and a 
respiration rate calculating stage 27 for calculating the 
number of respirations per unit time based on the data 
stored in the time-series-data storing stage 26. A frame 
interval setting stage 28 is connected to an output of the 
respiration rate calculating stage 27 for setting a frame 
interval a with which the inter-image subtraction stage 
24 performs inter-image subtraction, based on the 
result of calculation by means of the respiration rate cal- 
culating stage 27, and the frame memory selecting 
stage 23 is connected to an output of the frame interval 
setting stage 28 for selecting from the previous-frame 
memory 22 a previous frame image spaced an interval 
of a from a current image and outputting it to the inter- 
image subtraction stage 24. 

[0042] Now, the operation of ROI setting stage 10 will 
be described with reference to the flow chart of Fig. 4. 
[0043] First, in step S21 , images from the CCD cam- 
era 2 are stored into the original image memory 1 1 for a 
period of time equal to half the respiration period. In 
step S22, the subtracting and summing circuit 12 per- < 
forms subtraction at a predetermined subtraction inter- 
val (e.g., one frame), and sums up information of 
changes which are given by absolute values of subtrac- 
tions. An example of such changes of information thus 
summed is shown in Fig. 5, in which of dark portions t 
indicate image portions for which changes have been 
recognised during a time equal to half the respiration 
period. In step S23, the candidate-region extracting 



stage 1 3 carries out labelling processing on the informa- 
tion of image changes depicted in Fig. 5, and calculates 
an area and information of a rectangle enclosing the 
area for each label, thereby setting an ROI candidate. 
[0044] Although in this embodiment, a plurality of ROI 
candidates with labels having gradually decreasing 
areas from the largest to smallest have been prepared, 
a single candidate having a label of the largest area can 
instead be used, and if it is determined according to an 
1 optimal-ROI determination to be described later that 
this candidate is not optimal, the control process returns 
to step S21 so as to set a new ROI candidate once 
again. 

[0045] In step S24, the optimal-ROI determining stage 
14 determines an optimal ROI. Specifically, in this opti- 
mal ROI determination, an original image date related 
to the position and size of a rectangular region informa- 
tion set in the first candidate is obtained from the origi- 
nal image memory 11, thus providing a grey level 
histogram indicative of distribution of the number of pix- 
els at each grey level. The values of grey levels are 
determined according to 255 steps or degrees of grada- 
tion. An example of such a grey level histogram is 
shown in Fig. 6. In the grey level histogram, it is deter- 
mined whether a twin peak characteristic including two 
adjacent peaks at a short interval can be recognised 
and whether the area of the region (i.e., rectangular 
region) listed as a candidate is equal to or greater than 
a predetermined value. If these conditions are not satis- 
fied, the second candidate is taken as the next candi- 
date for its optimality determination according to this 
embodiment, as referred to above. 
[0046] Subsequently, if the conditions of determina- 
tion are satisfied in step S24, the position and size or 
rectangular area of the optimal ROI are stored in the 
ROI setting and storing stage 15, and thus the optimal 
ROI is finally set in step S25. On the other hand, if the 
conditions of determination are not satisfied, then the 
control process returns to step S21 , thus repeating the 
optimal ROI setting processing. 
[0047] After the optimal ROI has been set as 
described above, the respiration counting stage 20 or 
counts the number of respirations per unit time using 
the set ROI. 

[0048] The operation of the respiration counting stage 
20 will now be described with reference to Fig. 7. 
[0049] First, in step S31 , image data within the set 
ROI is fed from the CCD camera 2 to the previous-frame 
memory 22 and stored therein. Subsequently, in step 
S32, the inter-image subtraction stage 24 performs 
inter-image subtraction so as to provide an absolute 
value of subtraction for each pixel, which is then surface 
integrated by the surface integration stage 25 in step 
S33. 

[0050] The above processes are carried out in a time- 
series manner, as shown in step S34, and the data 
obtained during such processings are stored in the 
time-series-data storing stage 26, based on which data 
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the respiration counting stage 20 counts the number of avoid interrupting respiration counting for the time 
respirations per unit time in step S35. required to freshly set a new ROI thereby enabling res- 
[0051] Fig. 8 illustrates in real time the time-series piration counting to restart immediately. If it is again 
data obtained by the surface integrating stage 25. In this determined that the number of respirations per unit time 
diagram the respiration speed is shown by the absolute 5 freshly counted is abnormal, the abnormal respiration is 
value of subtraction, and as will be described, there immediately decided to be not due to inappropriate set- 
clearly appear a plurality of twin peaks each having a ting of the ROI thus enabling speedy recovery treat- 
pair of peaks indicative of exhalation and inhalation, ments to be taken, 
thus emphasising changes or variations despite noise 
interference.. 10 EMBODIMENT 3 
[0052] Here, the number of respirations per unit time 

is measured by counting the number of peaks exceed- [0056] Although in the aforementioned embodiment 1 , 
ing a constant or fixed threshold S H the respiration rate calculating stage 27 counts the 
[0053] In step S36, the frame-interval setting stage 28 number of peaks using the constant or fixed threshold 
automatically sets a frame interval in the form of the is S H , as shown in Fig. 8, the threshold S H may be varied 
interval of subtraction a based on the number of respi- for each peak. Fig. 1 0 shows in block form an exemplary 
rations per unit time as measured in such a manner as arrangement of a respiration rate calculating stage 27A 
to emphasise changes in the surface integration value in such a case. The respiration rate calculating stage 
(grey level), and outputs it to the frame memory select- 27A comprises a differential circuit 27a and a hold cir- 
ing stage 23 which then selects, based on the subtrac- 20 cuit 27b both connected to an output of a time-series 
tion interval a, a frame memory having a frame image data storing stage 26 (see Fig. 3), a subtracter 27c con- 
stored therein previously apart an interval of a frames nected to outputs of the hold circuit 27b and the time- 
from the current image, and outputs the data therein to series data storing stage 26, a setter 27e in which a pre- 
the inter-image subtraction stage 24. determined value is set, a comparator 27d connected to 
[0054] Here, it is to be noted that for the subtraction 2s outputs of the subtracter 27c and the setter 27c, and a 
interval a, a frame interval corresponding for example, counter 27f connected to an output of the comparator 
to half the respiration period is selected in order to 27d. 

emphasise changes in the grey level or darkness. In [0057] With the above arrangement, the differential 

view of this, the number of frames stored in the previ- circuit 27a differentiates the time-series data stored in 

ous-frame memory 22 is selected according to the ao the time-series data storing stage 26 to find its peak, 

number of frames per unit time and the respiration which is then output to the hold circuit 27b and there 

period. The previous-frame memory 22, in which the held therein as a peak value. The subtractor 27c calcu- 

data used for inter-image subtraction was stored, now lates the difference between the peak value and the 

becomes available for storing the next image data. In subsequent time-series data, and outputs the calcu- 

cases where the frame interval a is set according to the as lated difference value to the comparator 27d in which 

respiration period (or the number of respirations per unit the difference value is compared with the set value in 

time), as stated above, a change in the surface integra- the setter 27e. The comparator 27d generates an output 

tion obtained may be multiplied by 1/a, in order to indi- to the counter 27f when the difference value is greater 

cate the surface integration change at the same rate. than the set value. The counter 27f counts the number 
40 of outputs from the comparator 27d, thereby providing a 

EMBODIMENT 2 count of peaks. 

[0058] In the respiration rate calculating stage 27A, if 

[0055] In the aforementioned embodiment 1 , the res- a peak value varies, the threshold therein is accordingly 

piration counting stage 20 performs respiration counting changed to detect such a varying peak, so that the 

using the ROI set by the ROI setting stage 1 0, but in this 45 number of peaks can be counted accurately irrespective 

case, the ROI setting stage 10 may be constructed such of the magnitude of each peak, 
that it is always preparing for setting a new ROI during 

respiration counting by the respiration counting stage EMBODIMENT 4 
20. In this case, as shown in Fig. 9, an optimal-ROI 

determining stage 1 4A is provided with a temporal stor- so [0059] In embodiment 1 described above, the abnor- 

age stage 14a for storing an ROI each time the ROI is mality determining stage 30 is constructed such that it 

determined to be optimal by means of the optimal-ROI makes an abnormality determination based solely on 

determining stage 14A. Thus, for example, if the abnor- the number of respirations per unit time as counted by 

mality determining stage 30 determines the presence of the respiration rate calculating stage 27. However, there 

abnormality, the ROI setting stage 1 0 can immediately 55 may sometimes be a case where the abnormality deter- 

set again the optimal ROI stored in the temporal storage mining stage 30 cannot measure the true respiration 

stage 14a in place of the current ROI set in the ROI set- (i.e. respiratory movements) correctly, since the ROI 

ting and storing stage 15. In this manner, it is possible to once properly set departs from its original or intended 
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position due to movements of the object to be moni- 
tored, or the influence of noise affecting respiration 
counting. In view of this, to determine whether respira- 
tion is counted correctly, a frequency spectrum of a 
time-series data can be obtained for comparison with a 5 
frequency spectrum of a normally or correctly measured 
respiration. Such a comparison can be made using a 
statistical test of hypothesis. 

[0060] Fig. 1 1 shows in block form an example of such 
an arrangement. In this figure, a fast Fourier transform 10 
(FFT) stage 51 is provided at an output side of the time- 
series data storing stage 26 for generating a frequency 
spectrum of the time-series data output. A comparator 
52 compares the frequency spectrum of the time-series 
data output from the FFT stage 51 with a normal or cor- 15 
red frequency spectrum set in a frequency spectrum 
setting stage 53. This comparison is made, for example, 
in terms of the magnitude or strength of a predeter- 
mined frequency component. An abnormality determin- 
ing stage 30A determines, whether the respiration 20 
counting is normal based on the comparison of compa- 
rator 52. 

EMBODIMENT 5 

25 

[0061] Although in the above-described embodiment 
1 , the object detecting stage 40 detects an object com- 
ing into or the field of view of the CCD camera 2 upon 
setting an ROI by detecting a large change in time of the 
images produced by the camera 2, it may also detect 30 
the object going out of the field of view, again by detect- 
ing a large change in time of the images. A large change 
in time of the images can be detected, for example, by 
detecting a change in contrast or brightness of the 
images at predetermined time intervals by way of sens- 35 
ing correlation between the images or a change in the 
grey level histogram thereof. In this embodiment, then, 
an object detecting stage 40A may be provided with an 
image memory for the above purpose, but instead the 
original image memory 1 1 of the ROI setting stage 10 40 
can be used for investigating correlation between the 
images, or grey level histograms can be compared with 
each other at a predetermined time interval using a sta- 
tistical test of hypothesis so as to perform the above 
detection. Detection of the object to be monitored hav- 45 
ing gone out of the field of view is signalled to an abnor- 
mality determining stage 30B, so that it is possible to 
deal with the abnormal condition while discriminating 
respiratory abnormality from such an abnormal situa- 
tion in which normal respiration counting can not be so 
effected due to the absence of an object to be watched 
in the field of view of the camera 2. 
[0062] As described above, automatic setting of the 
ROI can provide the advantage of automatic accurate 
respiration counting for an extended period of time with- ss 
out requiring operator monitoring. In addition, exhala- 
tion and inhalation can quite easily be recognised on a 
monitor. 
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1 . A region-of-interest setting apparatus for respiration 
monitoring comprising: 

imaging means for imaging an object to be 
monitored; 

provisional-region setting means for extracting 
changed portions between successive images 
produced by the imaging means and selecting 
one of thus extracted changed portions as a 
provisional region; 

determining means for determining whether 
the provisional region set by the provisional- 
region setting means can be adopted as a 
changed region for respiration counting; and 
region-of-interest setting means for setting the 
provisional region as a region of interest for 
respiration counting when the determining 
means determines that the provisional region 
can be adopted as a changed region for respi- 
ration counting. 

2. Apparatus according to claim 1 , wherein the provi- 
sional-region setting means comprises: 

subtracted-image storing means for calculating 
absolute values of subtractions between every 
successive two of a plurality of frame images 
taken during at least half a respiration period, 
and for summing and storing the absolute val- 
ues of subtractions of the images thus obtained 
for every predetermined frame; and 

region-of-interest candidate setting means for 
calculating the position and size of each of the 
changed regions based on brightness informa- 
tion of each subtracted image stored in the 
subtracted-image storing means and setting at 
least the greatest among the changed regions 
as a provisional region. 

3. Apparatus according to claim 1 , wherein the provi- 
sional-region setting means comprises: 

subtracted-image storing means for calculating 
absolute values of subtractions between every 
successive two of a plurality of frame images 
taken during at least half a respiration period, 
and for summing and storing the absolute val- 
ues of subtractions of the images thus obtained 
for every predetermined frame; and 
region-of-interest candidate setting means for 
calculating the position and size of each of the 
changed regions based on brightness informa- 
tion of each subtracted image stored in the 
subtracted-image storing means and setting an 
optimal one of the changed regions succes- 
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sively from the greatest to the smallest thereof 
as a provisional region. 

4. Apparatus according to any of claims 1 to 3, 
wherein the determining means is arranged to s 
determine whether, in the provisional region set by 
the provisional-region setting means, a grey level 
histogram indicative of distribution of the number of 
pixels for respective grey level values includes a 
twin peak characteristic having twin peaks each 10 
equal to or higher than a predetermined value. 

5. Apparatus according to claim 4, wherein the deter- 
mining means is arranged to determine whether the 
area of the changed region is equal to or greater (5 
than a predetermined value. 

6. Apparatus according to any of claims 1 to 5, 
wherein, when the determining means is arranged 

to determine that the provisional region set by the 20 
provisional region setting means cannot be 
adopted as a changed region for respiration count- 
ing, the provisional region setting means is 
arranged to perform again the setting of a provi- 
sional region. 25 

7. Apparatus according to any of claims 1 to 6, further 
comprising region-of-interest setting starting-condi- 
tion determining means for detecting, as a condition 

for starting the setting of a region of interest, 30 
whether the object to be monitored has come into a 
field of view of the imaging means. 

8. A respiration monitoring system comprising: 

35 

imaging means for imaging an object to be 
monitored; 

setting means for setting a region of interest for 
respiration counting in images produced by the 40 
imaging means; 

change detecting means for calculating abso- 
lute values of subtractions between successive 
images produced by the imaging means in the 45 
region of interest set by the setting means and 
for summing the absolute values of subtrac- 
tions for every predetermined number of 
frames; and 

50 

counting means for counting the number of res- 
pirations per unit time based on a trend of time- 
series changes obtained by the change detect- 
ing means. 

55 

9. A system according to claim 8, further comprising 
condition determining means for determining, 
based on a frequency analysis of a trend of time- 



series data, whether the state of respiration is nor- 
mal. 

10. A system according to claim 8 or claim 9, further 
comprising existence determining means for deter- 
mining whether the object to be monitored has 
moved out of a predetermined imaging range. 

11. A system according to any of claims 8 to 10, 
wherein the respiration counting means is arranged 
to count the number of respirations per unit time by 
measuring the number of peaks in a change of a 
trend in the time series data, and wherein a thresh- 
old is used which is set for each peak for counting 
thereof. 

1 2. A system according to any of claims 8 to 1 1 , further 
comprising frame-interval setting means for setting 
the predetermined number based on the summed 
value calculated by the change detecting means or 
the number of respirations per unit time counted by 
the counting means in such a manner as to empha- 
sise changes in grey levels. 

13. A system according to any of claims 8 to 12, 
wherein the means for setting a region of interest 
for respiration counting in images taken by the 
imaging means comprises the region-of-interest 
setting means of the region-of-interest setting 
apparatus for respiration monitoring as defined in 
any of claims 1 to 7. 

14. A system according to claim 13, wherein the provi- 
sional region setting means and/or the determining 
means operate during respiration counting. 
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